Scalable Wide-Area Resource Discovery

This paper describes the design and implementation of S W O R D , a scalable resource discov ery serv ice for w ide-area distributed sy stems. S W O R D locates a set of machines matching user-specifi ed constraints on both static and dy namic node characteristics, including both single-node and inter-node characteristics. W e ex plore a range of sy stem architectures to determine the appropriate tradeoff s for building a scalable, highly -av ailable, and effi cient resource discov ery infrastructure. W e describe: i) techniq ues for effi cient handling of multi-attribute range q ueries that describe application resource req uirements; ii) an integrated mechanism for scalably measuring and q uery ing inter-node attributes w ithout req uiring O(n) time and space; iii) a mechanism for users to encode a restricted form of utility function indicating how the sy stem should fi lter candidate nodes w hen more are av ailable than the user needs, and an optimizer that performs this node selection based on per-node and inter-node characteristics; and iv ) w ork ing prototy pes of a v ariety of architectural alternativ es— running the gamut from centralized to fully distributed— along w ith a detailed performance ev aluation. S W O R D is currently deploy ed as a continuously -running serv ice on P lanetLab. W e fi nd that S W O R D off ers good performance, scalability , and robustness in both an emulated env ironment and a real-w orld deploy ment.